Modern machine learning models often combine multiple input streams of data to more accurately capture the information that informs their decisions. In multimodal machine learning, choosing the strategy for fusing data together requires careful consideration of the application's accuracy and latency requirements, as fusing the data at earlier or later stages in the model architecture can lead to performance changes in accuracy and latency. T o demonstrate this trade-off, we investigate different fusion strategies using a hybrid BERT and vision network framework that integrates image and text data. W e explore two different vision networks: MobileNetV2 and ViT. W e propose three models for each vision network, which fuse data at late, intermediate, and early stages in the architecture. W e evaluate the proposed models on the CMU-MOSI dataset and benchmark their latency on an NVIDIA Jetson Orin AGX. Our experimental results demonstrate that while late fusion yields the highest accuracy, early fusion offers the lowest inference latency. W e describe the three proposed model architectures and discuss the accuracy and latency trade-offs, concluding that data fusion earlier in the model architecture results in faster inference times at the cost of accuracy.

Approximate computing frameworks configure applications so they can operate at a range of points in an accuracy-performance trade-off space. Prior work has introduced many frameworks to create approximate programs. As approximation frameworks proliferate, it is natural to ask how they can be compared and combined to create even larger, richer trade-off spaces. We address these questions by presenting VIPER and BOA. VIPER compares trade-off spaces induced by different approximation frameworks by visualizing performance improvements across the full range of possible accuracies. BOA is a family of exploration techniques that quickly locate Pareto-efficient points in the immense trade-off space produced by the combination of two or more approximation frameworks. We use VIPER and BOA to compare and combine three different approximation frameworks from across the system stack, including: one that changes numerical precision, one that skips loop iterations, and one that manipulates existing application parameters. Compared to simply looking at Pareto-optimal curves, we find VIPER's visualizations provide a quicker and more convenient way to determine the best approximation technique for any accuracy loss. Compared to a state-of-the-art evolutionary algorithm, we find that BOA explores 14x fewer configurations yet locates 35% more Pareto-efficient points.